LARGE DEVIATIONS APPLICATION TO BILLINGSLEY'S 

EXAMPLE 



R. LIPTSER 



Abstract. We consider a classical model related to an empirical dis- 
tribution function Fn{t) = ^ 5Ilfe=i ^{?fc<t} of {^k)i>i - i-i-d. sequence 
of random variables, supported on the interval [0, 1], with continuous 
distribution function F{t) — P(^i < t). Applying "Stopping Time Tech- 
niques", we give a proof of Kolmogorov's exponential bound 

P( sup \F„{t) - F{t)\ >e)< const. e^"''= 

t6[0,l] 

conjectured by Kolmogorov in 1943. Using this bound we establish a 
best possible logarithmic asymptotic of 

P( sup n"\F„{t) - F{t)\ >e) 
te[o,il 

with rate ^i-2a slower than ^ for any a £ (O, |). 



1. Introduction 

Let (^fc)i>i be the i.i.d. sequence of random variables with values in the 
interval [0,1] having a continuous distribution function F(t) = P(^i < t). 

n 

Consider an empirical distribution = ^ /|^j.<f}.. A strong law of 

k=l 

large numbers for sums of i.i.d. random variables guaranties that for any t E 
[0,1], Fn{t) > F{t) and the Glivenko-Cantelli theorem also guarantees 

n^oo 

a uniform convergence sup^grg n \Fn{t) — F{t)\ > 0. 

For any fixed t, the rate of convergence, in n ^ oo, of \Fn{t) — F{t)\ is also 
well known from the Central Limit Theorem (CLT): {-^[Fn{t) — F{t)]}n^oo 
converges in law to a zero mean Gaussian random variable with the variance 
Fit)[l-F{t)]. 

From Theorem 16.4 of Billingsley (1968), it is known that the family 
{ {-^[Fn{t)—F{t)]^^^^Q ^ }n~*oo converges in law (in Skorokhod's and uniform 
metrics) to a zero mean Markov-Gaussian process X = (-^t)46[o,i] with a 
correlation function 

K{t,s) = F{s At)[l- F{sVt)]. (1.1) 

For F{t) = t, the limit random process X is known as "Brownian Bridge" 
defined as the unique solution of Ito's equation Xt = — Jq j^ds+Bt relative 
to Brownian motion Bf. In the general case, F{t) ^ t, the random process 
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X can be also defined as the unique solution of Ito's equation 

with Brownian motion Bt replaced by a Gaussian martingale Mt, EM| = 
F{t) (see Section El]). 

Once, Prof. A.N. Shiryaev has mentioned to participants of the Probabil- 
ity Seminar at the Steklov Mathematical Institute that in 1943 Kolmogorov 
conjectured the following rate of convergence in the uniform metric, 

P( sup \Fn{t) - F{t)\ >e]< const.e-"''% (1.3) 

\te[o,i] J 

a proof of which has never been published. 

In this paper, we give a version of Kolmogorov's exponential bound with 




It should be noted that neither Sanov's theorem (1961), [9] (see also Dembo 
Zeitouni, [2]) nor Wu's result (1994), [12], are not relevant tools for obtaining 
the Kolmogorov bound ()1.3p . since the Levy-Prohorov metric is involved 
in Sanov (1961) and Wu (1994). A crucial role in proving of (|1.3p plays 
"Stopping Time Techniques" . 

Unfortunately, we could not claim that (II. 3p is best possible bound even 
in a logarithmic scale. However, the Kolmogorov bound helps us to establish 
the following logarithmic asymptotics: for any a G (O, ^) and any T in a 
small vicinity of {1}, 

lim — i— logpf sup n"|F„(t) - F{t)\ > e) = -2e\ (1.4) 

We build the proof of ()1.4p based on Kolmogorov's bound and on a non- 
standard Large Deviations technique. A key for ()1.4p consists in choosing 
the rate i_„ slower than -. 

The paper is organized as follows. Section [2] contains auxiliary results 
from Stochastic Calculus useful for the asymptotic analysis of the random 
process {Fn{t) — ^(t))^^^^^ as n — > cx). Proofs of ()1.3p and (II. 4p are given 
in Sections [3] and H] respectively. The Large Deviations Principle result is 
formulated and proved in Section \K\ (Appendix) . 



2. Stochastic calculus applicability to Billingsley's theorem 

2.1. Xt as a Solution of (II. 2p . The limit random processes X = (^t)tg[o,i] 
is zero mean Gaussian with the correlation function defined in (jl.ip . By 
Theorem 8.1 of Doob (1953), the gaussianity of X jointly with an obvious 
property of the correlation function, 

K[u, u) 
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enable us to claim that X is Markov process with respect to a minimal 
filtration {■^i^)t<^[o^i\ generated by X. Then for s < u < t, 



X, 



Xt 



Y 1 K{t,u) ^ _ Xu 
■^u I — wTTTTT 



l-F(t) \1-F{t) l-F{t)K{u,u) " l-F(n)' 

In other words, the Gaussian random process Nt = i^F(^t) square 
integrable martingale, i.e., a process with orthogonal increments (so, with 
independent increments too). Hence, its predictable variation process {N)t 
coincides with EA^^^ ^ ^^^M^ = j^^. 

Therefore, the process = Jq*[1 — F{s)]dNs is the Gaussian martingale 
with 

{M)t = - Hs)?d{N)s = - ^(^)]'^(t^^) = 



Finally, the Ito equation (II. 2p is derived by applying the Ito formula to 
Xt = [l- Fit)]Nt. 

n 

2.2. Counting Process ^{^k<t}' Without loss of generality we shall 

ik=l 

assume that all ^^'s are defined on a probability space {fi,J-, P). Denote 

• = {^t)o<t<i the filtration generated by Ij^^^^}, 

• •^=Vte[o,i]^* 

and assume also that the general conditions for these filtrations are fulfilled. 

The random process ^{^^.<t\ has piece-wise constant and right continuous 
paths with only one jump of the unit size. Thus, (/{^j,<f}, ^/^')tg[o^i] is a 
counting process with continuous (!) compensator (^t )tg[o,i]) 

'~Jo 1 - Fis) -Jo I- Fis) ^^^^^ 
(see, e.g., formula (18.23), Section 18.2 in [6j). Set M/= = I{^^<t} - A^. 
It is well known (see, e.g., Ch. 18 in [6]) that (M/^, ^/^)igjo,i] is a square 
integrable martingale with paths from the Skorokhod space IJ[o,i] and its 
predictable quadratic variation process {M^)t = A^. The joint independence 
of {£,k)k>i implies that {(/{^^<t}, =^t)tg[o,i]}fc>i are counting processes with 

n 

disjoint jumps. Set I" = E %fc<t}- Then, (I", =^t)tg[o,i] is a counting 

fc=i 

process with the corresponding compensator, 

^t=pA^ = nl^'-^^dFis) (2.1) 

or, equivalently, (I" — A", =^t) j^jq ^ is the square integrable martingale with 
the predictable variation process A". Two other martingales are related to 
{I2-A2,^t),^[0,i] ■■ (M?,^t)t6[o,i] and (M^ , ^*)ie[o,i], where 

1\ 



MJ^ = 4= (ir - and Mj^'" = -^M]^, a G 



n 



n2 



"•2. 
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with predictable variation processes respectively: 

(M") = iAr and (M"-)* = -^A^. 



2.3. Functional Central Limit Theorem for M^. 

Theorem 2.1. The family of martingales { (Afr)tg[o ij) In^oo (converges in 
law (in Skorokhod's and uniform metrics) to a Gaussian martingale with 
{M)t = F{t). 

Proof. In view of the function F{t) is continuous, the Gaussian martingale 
Mt is continuous too. Then, by Theorem 2, Ch. 7, §1 of Liptser-Shiryaev 

(1989), [5], the desired statement holds true provided that (M") > 

^ ' * n— >oo 

V t G [0, 1]. The latter holds since 

1 1 " ft ^ — T 

\ It ^ t nf^Jo l-F{s) ^ ' 

and, in the case under consideration, the strong law of large numbers for 
sums of i.i.d. random variables implies 

1 J 

lim / M"\ = E / ^^\-"U f(s) = F(t) a.s. V t G [0, 11. 

n-»oo \ / 1 Jq 1 — F{s) 

□ 



2.4. Semimartingale Decomposition of Centered Empirical Distri- 
bution. Set 



n 



■■[Fnit)-F{t)], aG 



0, 



1 



Lemma 2.1. For t £ [0,1), 



(i) 



X ' 1 

t 



(ii) xr = ^[l-F{t)][- 

Jo ^ 



(iii) X^^ 



(iv) XI 



1 

— 1 




1 




— 1 






1 


1 



FisY 

rt 



[1 - F{t)] 



[l-F{s)f 



dF{s)]- 



-M" 



(2.2) 



l^^^jj, where for any function {xt)t(i[o,i\ from 



the 



Skorokhod space ©[o,!], 



v|;(x[o,j]) =Xi-[l-F(t)] 



;dF{s) 



is continuous function in the uniform metric on [0,1]. 



Proof, (i) From (j2.2p and the definition of A" and M", it follows that 

n 

Xr = 7I^E i^t - Fit)] + ^M?. Consequently, 
fc=i 



1 " 



fc=i 



fc=l 



fc=l 



dFjs) 
1 - F{s) 



F{t) 







l-F s) 



dF{s) - F{t) 



n^[Fn{s) - F{s) 



7o 1 



F{s) 



■dF{s) 



1-F(s 



-dF{s) 



(ii) This formula describes the unique solution of Ito's equation from (ii) 

(iii) The Ito formula j^j^ = /g + jY^^^f^dF{s) and (ii) pro- 
vide 



-4^^t = - Fit)] 

= 4^i^-m] 

772 
7-n,a 



Mr 



1 - Fit) 



+ 







i-Fis) 7o 



xr+^[^-Fit)] 



772 



(iv) ^ix\Qf]) is nothing but (iii) with xt replaced by ^— M". A desired 
continuity of v|/ follows from 

sup 14 - x'/l < £ ^ sup |M/(x(o - i)| < 2e. 



te[o,i] 



te[o,i] 



□ 



2.4.1. An Alternative Proof of Billingsley's Theorem. For a = ^, 

write '5 = ^[Fnit) - Fit)] . 

Lemma 2.2. The family {{X^'^)t£[o^i])} _^ converges in law iin Sko- 
rokhod's and uniform metrics) to the continuous Gaussian process iXt)t(z[(j^ij 
defined in (jl.2p . 



Proof By Lemma 0(iv), Xf'^ = \l/(-^M|J) ^j) and by Theorem EH 



Now, by applying the Ito formula to Xt := U^(M[o,j]), we make sure that 
solves dO]). ' □ 
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3. The Kolmogorov bound 

In this section, we show that 



P( sup \Fn{t)-F{t)\ >e 



< 2exp 



log 1 + 



32 



1 4 , / 



(3.1) 



Since Fn{t) and F{t) are increasing functions and F(t) is continuous, the 
following upper bound with a free parameter T £ (0, 1) holds: 

sup \Fn{t) - F{t)\ < sup \Fn{t) - F{t)\ + sup - F{t)\ 

te[o,i] t£[o,T] te(T,i] 

< sup |F„(i)-F(t)| + |l-F(T)| + |l-F„(T)| 

< sup |F„(i)-F(t)|+2[l-F(r)] + |F(r)-F„(T)| 

te[o,T] 

<2| sup |i^„(t)-F(t)| + [l-F(r)]|. 

"^te[o,T] 

A choice of T with 1 — F{T) = | guarantees a useful upper bound 
p( sup - F{t)\ > < Pf sup \Fn{t) - F{t)\ > ^ 



te[o,i] 



iG[0,T] 



By Lemma |2. II ((iv)) with a = 0, we find that 
sup \Fn{t) - F{t)\ = sup 

tG[0,T] ie[0,T] 

< ^ sup |Mr|(l+ sup [1-F{t)] f - 



dF{s) 



< 



[0,1] 

sup I I 

" te[o,T] 



F{s)Y^ 



and the following upper bound: 

P( sup \Fnit) - Fit)\ > e) < P( sup ^IM^I > |). 

te[o,i] t&[o,T] o 

Now, we shall combine "exponential martingale" and "stopping time" 
techniques. With A > 0, let us introduce the exponential martingale 



U = exp ( AmJ' 



n 



n 



(3.2) 



relative to the filtration {'^t)te[o,i]- It is well known that any exponential 
martingale is a supermartingale too, that is, (34, '^t)te[o,i] is the nonnegative 
supermartingale with E^,- < E30 = 1 for any stopping time r w.r.t. the 
filtration (^t)[te[o,i]] • 

We choose two stopping times, 



= inf |t < T : ±^M1 > |}, inf(0) 



00, 
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and use them for obtaining the following bound: 
1 



P( sup |Mr|>^)<2 



max 



P r" < oo , P r!! < oo 



In order to find an upper bound of P(t-|_ < cxd), write 

1 > ^Ir^ > E/{r+<oo}3r+ = E/|^^<oo} exp (^X-^M"^^ - 



X 
6" 



1-^ 

n 



A" 



> P(t+ < oo)exp (A 



A 
6" 



By dZID, < j^f^ = f, so that 
1 > P(r+ < oo)exp (a 







6" — 1 


t) 


n. 





or, equivalently, P(t" < oo) < exp ( — { A| — [e™ — 1 — ^] ^}) • Since A is an 
arbitrary positive parameter, we can set A as A* = argmax^^g ~ \_^~ ~ 
1 — ^] ^} = n log (l + 1^) , in order to obtain 

P{tI < oo) 



< exp 



A*- 



1 



A* 



n 



t}) 



exp 



n 



8l ^ V 3 



1 4 , / 
l} + -log(l + 35 



The proof of the upper bound P(r" < oo) < exp ( — ?^[|{ log (l + ||) — 
l} + I log (l + §1)]) is similar. 

Therefore, © holds. □ 

4. The proof of (fTTll) 



Recall that Xi*"'" = n° [F„(t) - F(t)] (see 
Theorem 4.1. For any a G (O, ^) and any T in a small vicinity o/{l}, 



lim 



^ 2,logP( sup n"|Xn>el =-2e^ 

i6[0,T] 



Proof. By Theorem I A . 2 1 (Appendix) the family {(n"X"'"))^g[o,T]}n-»oo obeys 
the large deviations principle in the Skorokhod space II5[o,i] relative Sko- 
rokhod's and uniform metrics with the rate and the rate function 



Jt{u) 



Ut + 



1 - F{t) 



oo, 



2 "0=0 
'dF(t) dut=utdF{t) 

otherwise. 



Since paths of (^"'")tG[o,T] with property { sup^g[Q n"|X"'"| > e} form a 
closed set 

{uo=0 \ 
U G ]D)[o,T] : eiu)=mf{t<T:\ut\>e}<T I , 
nt=0, t>e{u) ) 
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in accordance with the large deviations theory, 

hm ^ log P ( sup > e) < — inf Jt{u). 

n^con^ ^" \e[o,T] «6C 

A minimization procedure of Jt{u) in u € C automatically excludes from 
consideration all functions (wt)i6[o,T] with Jt{u) = oo. Consequently, 

lim i-2a P( sup n"|X"'"| > e") < 



n— >oo /i^ 



tG[0,T] 



2 f-TAe{u) 

- inf / -it^ + 



1 - F{s) 

Denote wt = ut + i^F{t) • '^'^^^^ Je{u){'^) = ^ /o^''"^ WfdF{t), and 

^t = - [ , , dF{s) + / t < e(u). 

This integral equation obeys the unique solution 

u^^eiu) = [1 - F{t A e{u))] I ^_L. dF{s). 



2 



dF{s) 



The assumption 9{u) < T implies = Hence 



e' = [l- F{9{u))r [ 3Trp^'^^(^) j • (4-1) 

Now, we apply the Cauchy-Schwarz's inequality, 

T^r^^^)^l rap/ « ^'-'^ 

F{6{u)) 



1 - F{9{u 



■2Je(„)(u), 



2 

transforming (14. ip into the lower bound: Je(u)iu) > 2F(6'(m))[i-F(6>(m))] • 
sume for a moment that there exists such that F{9{u*)) = ^. Then 
the following lower bound Jg(^jj_*^{u*) > 2e^ is valid. This lower bound is 
attainable, Je[u*){'^*) = provided that the Cauchy-Schwarz's inequality 
in (14. 2p becomes the equality. The latter holds true if w* , related to {ill ) ■> 
is in a proportion to jzj^i w* = and there exists a constant I* 

such that ( /g^^"*^ j::^(iF(s))^ = Ae'^. The existence of /* = 2e is verified 
directly. 

Thus, the upper bound is valid: 

lE^ ^^T^logPf sup > e)< -2e^ . 

n^oo n te[o,T] 



In order to complete the proof, we have to prove the following lower bound 
1 

n^oo ^ t6[0,T] 



lim -7Z2^1ogP( sup n"|Xj"'"| >e) > -2e^ 



Formally, one may apply 

1 



lim ' logP( sup n^'lX^'"] >e 



2 i-The{u) 



> — inf 

2 MeC° 







u 

Us + 



1-F{s) 



2 



dF{s), 



where C° is an interior of C. However, C has an empty interior. Fortunately, 
the proof of the upper bound gives us a hint: F((9(u*) = i. Choose T* with 
F(r*) = i and use an obvious inequality: 

P( sup > e) > pfn"|X^f I > e). 

Hence, only a lower bound lim ^^20, log P(n'^|X^i"| > e) > — 2e^ has to be 

n— too 

proven. The latter is verified with the help of the large deviations principle 
for the different family {Xy«"}„^oo- 

n 

Since = E [%<T.}-i^(T*)] with (/{^,<^.j -F(r*)),>i 

being the i.i.d. sequence of zero mean random variables having the variance 
F{T*)[1 — F{T*)] = J, the large deviations principle for this family is well 

known and has the rate ^i-ia and the rate function I{v) = 2F(t*)[i-F{t*)] ~ 
2v^. Therefore, 

1 



lim ^^logP(n"|JSs:"f I > e) = - inf I(v) = -2e^ . 



□ 



Appendix A. Large deviations principle for X"'" 



By ([221), ^r'" = - /o t5St^-^(*) + ^TT^"- ^ comphcated structure 



of the martingale {JsAfi , ^t)t&[o,i] does not allow us to apply Freidlin and 
Wentzell's (1984), g], or of Wentzeh's (1986) [H] results. 

On the other hand, by Theorem 12.11 the family {(M")tg[o^i]}n-»oo con- 
verges in law to Gaussian martingale {^t)t&[o.i] with (M)^ = F{t). Notice 
also that the family {( i_ ^t)t&\Q iljn^oo is in a framework of Freidlin and 
Wentzell (1984). So, it obeys the large deviations principle with the rate 



and the rate function 



M0 = 



ilidFii] dut=utdF{t) 

^00, otherwise. 



Theorem A.l. For any a G (O, ^) and any T in a small vicinity of {1}, 
the families 

i(^-mt] \ and \(^-mA \ 

L V^2~" / te[0,T] J n-too l-V^a-" / te[0,T] J n.-too 

share the same large deviations principle. 
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Proof. Hereafter, A(s) is a bounded measurable function. 

Since Jq A(,s)(iMs is a continuous Gaussian martingale with the pre- 
dictable variation process —j ^ J^X^{s)dF{s) =: 24"'°(A), the function 
S'^'°'{X) is the Laplace transform of Jq A(s)(iMs. Moreover, a random 



n2 



process 3^ 



exp (— rr^ Jo ^{^)'^^s — log(o"'"(A)j is a martingale. In 



n2 



the case of Jq A(s)(iM", an explicit formula for the Laplace transform 



is unknown. However, a random process (f"'"(A) = exp ( [e"^"" — 1 — 

^/■f I ] d A" j "exponentially compensates" /q A(s)(iM" up to a mar- 

tingale in a sense that a random process 3"'° = exp ( — Jg* A(s)(iM" — 

(p^"'°(A)^ is a local martingale (the latter is verified by applying the Ito 
formula) . 

By a terminology of Puhalskii (1994, 2001), 4"'"(A) and ^"'"(A) are 
referred to as "Stochastic Exponentials" related to the families 



{{4. 



-M, 



t6[0,T] 



and 



1 



1 

112' 



-Mr 



te[o,T] 



respectively. 

A role of stochastic exponential is revealed in Puhalskii (1994, 2001). In 
our setting the Puhalskii result states that the above-mentioned families 
share the same large deviations principle provided that for any rj > and 
any bounded X{t), 



lim 



1 



l-2a 



log P ( sup n 

te[o,T] 



l~2a 



4"'"(A) 



-00. 



(A.2) 



We finish the proof by verification of (|A.2h . Taking into account 

1 - Fnis) 



1-F{s) 



■dF{s) 



(see ()2.ip ). write 



n 



,1-2q 



n 



l-2a 



log 



n 



,l-2a 



en 



Ms) 



Ms) 



X{s) 



n 



en 

L 

t r- \2 



n 



l-a 



2n2(i-") 



dF{s] 



n 



X'{s) l-Fn{s) 
n- 



1 - Fn{s) 

l-F{s) 
dF{s) 



dF{s) 
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Since 



,l-2« 



Xis) XHs) 



2n2(i-") 



n- , aF[s) 



< 



1-F{s) 

const.—; / ^, , dFls) = const.— loe ^ 

ni-" Jq 1 - F{s) ^ ' ni-" ^ 1 - F(r) 



and 



n 



,l-2a 



< 



^A2(s)|F„(s)-F(5M 







l-F(s) 



< const, sup \Fnis)-F{s)\ / ^dF{s) 

se\o.T] Jo 1 - F{s) 



1 



= const, sup -F(s)|log- , 

se[o,T] - -f* U J 

we shall analyze an upper bound of the following inequality: 



n 



l-2a 



sup log^ 



< const, log 



1 



1 - F{T) 



sup \Fn{s)-F{s) 
n se[o,T] 



Obviously, ()A.2p is valid if 

1™ n"!^"^! '^^P \Pn{s) - F{s)\> 7] 1-) = -oo. 

For fixed r?, let us choose a number no such that < S and all n > no. 
In this scenario it remains to show that 



lim 



n—*oo n 



^^?( sup \Fn{s)-F{s)\ > l 



-oo. 



The latter heavily uses Kolmogorov's bound: 



1 



n 



l-2a 



logP sup \Fn{s)-F{s)\ > i 



se[o,r] 



<i^-n- 



n 



l-2a 



^|log(l + ^)-l| + -log(l + ^)l') 
16 1 128/ / ?7 128/Jy 



— oo. 



□ 



Theorem I A . 1 1 implies the following result. 



Theorem A. 2. For any a G (O, ^) and any T in a small vicinity of {1}, 
the family {(X"'")igjo,T]}n-*oo obeys the large deviations principle in the 
Skorokhod space Bjo^t] relative Skorokhod's and uniform metrics with the 
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rate speed -^h^ and the rate function 




(ut+ \A dt, ^ du,=u,dF{t) 

V I- Ft)) rr(ut+^^)^dF( 



otherwise. 

Proof. By LemmaEH = vl;(^^!— . Hence and by Theorem|XT] 
the family {(^('^'")tg[o,T]}n-^oo shares the large deviations principle with the 
family ^J/f^^—Mroti) 

Hence, by the contraction principle of Varadhan (1984) and (jA.ip Jt{u) = 
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